How Design Science Research Helps Improve Learning Efficiency in Online Conversations

In this design science research paper, we report on our constructing and evaluating an attention-guidance system that we integrated into a computer-supported collaborative learning system. Drawing on social constructivist literature, our proposed design focuses on attracting, retaining, and, if necessary, reacquiring users’ attention on task-relevant information in online collaborative literature processing. The investigation involved an experiment across two sections of students in a human-computer interaction course. Results show that the new design allowed users to consistently reflect and evaluate the content of a text as they capitalized on one another’s reasoning to resolve misconceptions. Moreover, we found that the new system increased users’ perceptions of learning. However, the difference in knowledge gain scores was marginally significant and represented a medium effect size. Interestingly, we found that the attention-guidance system supported more efficient learning. Finally, we discovered that task-oriented reading of text, revisions of incomplete or incorrect ideas, and perceptions of learning mediated the relationship between software system and learning efficiency. We discuss the theoretical and practical implications

Cluster Analysis in Online Learning Communities: A Text Mining Approach

This paper presents a theory-informed blueprint for mining unstructured text data using mixed- and multi-methods to improve understanding of collaboration in asynchronous online discussions (AOD). Grounded in a community of inquiry theoretical framework to systematically combine established research techniques, we investigated how AOD topics and individual reflections on those topics affect formation of clusters or groups in a community. The data for the investigation came from 54 participants and 470 messages. Data analysis combined the analytical efficiency and scalability of topic modeling, social network analysis, and cluster analysis with qualitative content analysis. The cluster analysis found three clusters and that members of the intermediate cluster (i.e., middle of three clusters) played a pivotal role in this community by expressing uncertainty statements, which facilitated a collective sense-making process to resolve misunderstandings. Furthermore, we found that participants’ selected discussion topics and how they discussed those topics influenced cluster formations. Theoretical, practical, and methodological implications are discussed in depth

Health Information Text Characteristics

Millions of people search online for medical text, but these texts are often too complicated to understand. Readability evaluations are mostly based on surface metrics such as character or words counts and sentence syntax, but content is ignored. We compared four types of documents, easy and difficult WebMD documents, patient blogs, and patient educational material, for surface and content-based metrics. The documents differed significantly in reading grade levels and vocabulary used. WebMD pages with high readability also used terminology that was more consumer-friendly. Moreover, difficult documents are harder to understand due to their grammar and word choice and because they discuss more difficult topics. This indicates that we can simplify many documents by focusing on word choice in addition to sentence structure, however, for difficult documents this may be insufficient

Integrating Learning Analytics to Measure Message Quality in Large Online Conversations

Research on computer-supported collaborative learning (CSCL) often employs content analysis as an approach to investigate message quality in asynchronous online discussions using systematic message-coding schemas. Although this approach helps researchers count the frequencies by which students engage in different socio-cognitive actions, it does not explain how students articulate their ideas in categorized messages. This study investigates the effects of a recommender system on the quality of students’ messages from voluminous discussions. We employ learning analytics to produce a quasi-quality index score for each message. Moreover, we examine the relationship between this score and the phases of a popular message-coding schema. Empirical findings show that a custom CSCL environment extended by a recommender system supports students to explore different viewpoints and modify interpretations with higher quasi-quality index scores than students assigned to the control software. Theoretical and practical implications are also discussed

Social Capital and ICT Intervention: A Holistic Model of Value

Background: Despite increasing popularity of Social capital, the relationship between social capital and ICT often appears to be an ambivalent one. Existing information systems (IS) literature presented various frameworks and theoretical foundations to facilitate the study of this concept, yet several contradictory findings have been reported indicating a significant knowledge gap in this domain. Current research adopts a holistic approach to address this knowledge gap by answering “How does social capital generate value or benefits in an ICT intervention?” Method: Current research employs a systematic literature review coupled with a grounded theory method to investigate proposed research questions. Results: Primary contributions of the current research include (1) the identification of contextual relationship between contextual factors and social capital dimensions, and (2) development of a holistic model of social capital driven benefits during ICT intervention where the ‘enablers’ and the ‘drivers’ of benefit have been identified. Conclusions: Identification of distinct roles and value-drivers related to social capital will help IS researchers in explaining “how and why” benefits are achieved while employing a social capital lens. Availible at: https://aisel.aisnet.org/pajais/vol11/iss4/3

Development of a Reading Material Recommender System Based On Design Science Research Approach

Using design science research (DSR), we outline the construction and evaluation of a recommender system incorporated into an existing computer-supported collaborative learning environment. Drawing from Clark’s communication theory and a user-centered design methodology, the proposed design aims to prevent users from having to develop their own conversational overload coping strategies detrimental to learning within large discussions. Two experiments were carried out to investigate the merits of three collaborative filtering recommender systems. Findings from the first experiment show that the constrained Pearson Correlation Coefficient (PCC) similarity metric produced the most accurate recommendations. Consistently, users reported that constrained PCC based recommendations served best to their needs, which prompted users to read more posts. Results from the second experiment strikingly suggest that constrained PCC based recommendations simplified users’ navigation in large discussions by acting as implicit indicators of common ground, freeing users from having to develop their own coping strategies

Towards a Sentiment Analyzing Discussion-board

In this paper we present the design and construction of a sentiment analyzing discussion board, which was used to support learning and interaction within an existing online social networking (OSN) system. More specifically, this research introduces an innovative extension to learning management software (LMS) that combines real-time sentiment analysis with the goal of fostering student engagement and course community. In this study we perform data mining to extract sentiment on over 6,000 historical discussion board posts. This initial data was analyzed for sentiment and interaction patterns and used for guiding the redesign of an existing asynchronous online discussion board (AOD). The redesign incorporates a sentiment analyzer, which allows users to analyze the sentiment of their individual contributions prior to submission. Preliminary results found that the proposed system produced more favorable outcomes when compared to existing AOD software

The Effect of Anchoring Online Discussion on Collaboration and Cognitive Load

Online discussions provide an opportunity for collaborative construction of meaning through peer to peer dialogue. The aim of this study is to develop an understanding of cognitive load as a factor supporting or inhibiting students’ participation in online asynchronous discussions. We employ cognitive load theory as a theoretical perspective, applying it not only to participants’ cognitive load but also to their collaboration load. Through an experimental study, we confirm that anchoring discussion leads to more task-oriented communication and less need for social and planning comments, which leaves more time and effort for the creation of elaboration and evaluation of ideas. Furthermore, anchoring discussion leads to more efficient communication as it reduces cognitive load involved in correctly interpreting messages

Instructor versus Peer Attention Guidance in Online Learning Conversations

This paper reports a theory-driven experimental study for designing and evaluating two different forms of attention-guidance functionalities integrated into an anchored-discussion system. Using social constructivism as a motivating theory, we constructed a theoretical framework that emphasizes the importance of students’ attention allocation in online learning conversations and its influence on message quality and interaction patterns. The development of the functionalities, named faded instructor-led and peer-oriented attention guidance, aimed to direct students’ attention toward instructional materials’ central domain principles while offering them an open learning environment in which they could choose their own topics and express their own ideas. We evaluated the functionalities with heat map analysis, repeated measures general linear model analysis, and sequence analysis to assess the utility of the developed functionalities. Results show that attention guidance helped students more properly allocate their attention in online learning conversations. Furthermore, we found that the improved attention allocation led to better quality of students’ online learning conversations. We discuss implications for researchers and practitioners who wish to promote more fruitful online discussions

UNDERSTANDING THE PARADOX OF MENTAL EFFORT IN ONLINE LEARNING CONVERSATIONS

This study investigates inquiry-based interaction and learning outcomes mediated by two types of artifact-centered discourse environments. The study aims to promote social construction of knowledge by optimizing the division of mental effort between pragmatic and semantic grounding activities. We present a theoretical research model by combining social constructivism, grounding theory, and cognitive load theory. We carried out a quasi-experimental study using survey instruments, content analysis, sequential analysis, and knowledge tests for a holistic approach to understand the paradox of mental effort in online learning conversations. The primary finding of this study is that a linked artifact-centered discourse environment facilitates pragmatic grounding activities to attain a common ground in online learning conversations. Additionally, less need for pragmatic grounding activities leaves more room for semantic grounding activities. Finally, more semantic grounding activities lead to a deeper understanding of the learning material